Best Practice Guide - Audiences, Tracks, Events and Polling

The first steps to any successful campaign are design and planning.

In advance of building any campaign, consider the following:

• Is this a list or journey based campaign?
• How often will the audience change?
• What event response times are required?
• When will the campaign start and end?

A design is required before the rest of the information in this document is applied.

NOTE:

See the Best Practice Guide - Conceptual Campaign Design for further advice on campaign design.

Audience refreshes, for example, can take some time to calculate and if the audience has not changed this is unnecessary processing, so understanding this process is relevant to campaign design.

Carefully managing polling with tracks and events is vital to ensure optimal response times for campaigns and to ensure that the system is not overloaded with unnecessary polling. Having a polling strategy that is too frequent can significantly influence system performance. This guide provides advice on how to approach polling strategy.

The document is produced in line with functionality in Campaign Manager and Engine.

There are two campaign types, Track Segmentation and Grid Segmentation. The main features of these are as follows:

Grid Segmentation

• High volume cell based campaign
• Supports complex waterfall campaign segmentation
• User-friendly grid User Interface for ease of count review
• Locked/static audiences with dynamic suppressions
• Export to Excel for management sign-off
• Flexible use of tactics and variables
• Suited to a more ad-hoc single phase execution
• Suited for short life-cycle campaigns
• Can be used as a start of a Customer Journey/Track Segmentation campaign if required

Track Segmentation

• Suited to journey based event driven engagement campaigns
• Flexible track based execution
• Multi-phase, reactive with dynamic audiences
• Aimed as a strategy based campaign activity
• Must be approached in a conceptual design initially
• Requirement for a robust testing phase
• More Marketer team/Senior management led

A campaign’s audience is simply a combination of the Inclusion Segments minus the Exclusion Segments as defined on the Audience tab of the campaign document.

KEY FACT:

The key aspect of the audience refresh is that it has the potential to re-calculate as part of every track job. If a recipient no longer meets the audience criteria, they will fall out of the campaign. It is important to construct the audience query correctly. For example, a Welcome Journey type campaign for new Customers, and an Inclusion Segment of “Days since sign-up < 3”. After 3 days the Customers who entered on Day 1 would fall out of the audience and no longer be processed in any later tracks of the campaign, which presumably would be undesirable. In this case, an audience query is required that would ensure these Customers stay in the campaign for the duration of the campaign. Meaning that the audience consists of existing Customers and Track logic is used to define sign-ups in the last 3 days. Alternatively, as part of campaign design, define the Welcome Journey as lasting for 90 Days and the audience includes people whose Days since sign-up < 90. This means that after 90 days they would fall out of the campaign regardless of their track position.

An audience refresh is a re-run of the calculation that creates the audience. If new recipients are found, they are added to the campaigns list of recipients. If any existing recipients no longer meet the criteria of the section, they are flagged as no longer included in the campaign and no further processing will occur on those recipients.

KEY FACT:

In the background the Campaign Manager system stores a table, known as campaign state, for each campaign. This table consists of a unique list of recipients. If new recipients are found they are added to the campaign state, if any existing recipients no longer meet the criteria of the selection, they are flagged as no longer included in the campaign and no further processing will occur on those recipients. This means that if a recipient falls out of the audience, no further action is taken against them, but they will remain in their current track position, so if they are re-included for any reason, they will pick up where they left off and not re-enter in the START track.

Campaign state can be viewed on a track by track basis on the Overview tab of the campaign document using the “State” tab.

In a campaign’s Audience tab, there is the option to define the frequency of audience refresh:

The default option is On Start, which means the audience calculation will run only when the campaign starts or re-starts. For example, after a load process, to see if any new recipients have entered. If the audience data is going to change during the day, for example with new signs ups being processed, the Always option should be selected. This means the audience will refresh on every track poll.

KEY FACT:

The above setting is the audience calculation only and will not restrict any recipients who are already in the audience, so should not be considered as limiting for intra-day campaign activity on known recipients. This option does not relate to suppressions and these will always refresh on a track poll to ensure they are suppressed immediately.

If a campaign requires an audience that after initial creation is locked for the duration of the campaign, select the Lock Inclusions After Start check box in the Audience tab.

KEY FACT:

This occurs on first track poll and NOT on campaign start. This is audience inclusions only and suppressions will always re-calculate on every track poll.

Most actions in Campaign Manager work on a 'pull' system of initiation. That means that activity does not commence immediately when a data change occurs by having that data 'pushed' into processors, but instead the architecture relies on making regular checks for these changes and then ‘pulling’ those changes back into the system for processing and execution.

The process of making these regular checks is called ‘polling’ and is set on a track-by-track basis using tools called time items.

In general, the more frequent the polling, the closer the 'pull' mechanism will approximate to a 'pushed' one, but at the cost of more frequent and probably redundant checks.

KEY QUESTIONS

How often will your data change? What are the actions that would introduce new recipients to the audience of a campaign or move them to a new track, and how frequently do you want to process them? Are data changes purely driven off data via the regular data load? In some cases, they will not change intra-day removing the need for aggressive Polling or even intra-day audience refreshes. Are email or external events driving recipients through your campaigns? If so you may decide to use Event polling and processing to drive track transitions. This guide will discuss this point more later.

KEY FACT:

This occurs on first track poll and NOT on campaign start. This is audience inclusions only and suppressions will always re-calculate on every track poll.

In Campaign Manager, there is a back end process called the SQL Job Service, which is best thought of as the process in charge of looking for things to do.

All polling activity begins as a result of the SQL Jobs Service performing a check every minute, across all the campaign tracks and event sources in all of the campaigns, to see if any are due to be processed.

When it checks the campaigns, if there are tracks to be processed, a job is added to a queue. Whether a track is due to process depends on the time item used and therefore on the user specified polling or scheduling on the time item.

KEY FACT:

Campaigns will always process from left to right so on start of a new campaign, the START track will always be first. This affirms the logical presumption that campaign design will occur from left to right in the campaign, and top to bottom within specific tracks.

A second process called Alchemy DocumentScheduleService runs through and executes jobs when they are added to the queue.

KEY FACT:

The more jobs there are in the queue, the longer each job takes to get to he top of the queue. System performance can be optimized by not adding too many jobs to the queue if you do not need them. The default polling on a “Do Immediately” Track is 600 seconds (10 minutes). This is fairly aggressive, so consider if this is appropriate. Your system administrator can change the default Setting if required.

This is a very important question when considering campaign design. Each Track in a campaign can only have one time item, so choosing the right one is of paramount importance.

KEY FACT:

The Time Item is a Track container that means that anything you add to the track is contained in the Time item. If you set up a Track then delete the time item, you will delete all other tools in the Track, so choose the time item first.

By default, the time item in every track is Do Immediately, as seen here:

Click the small cross icon if you want to remove the current time item.

Click the larger drop down icon and select a time item for the Track.

In the screen shot below, the Start Track has the Do immediately time item, Track 1 has On a schedule, Track 2 has a specific date and Track 3 displays the selection drop-down:

When you use the Do immediately and After a Period time items, you need to define how often the system should check for new activity, this is the user defined Polling frequency. Click on the time item. Click the Attributes tab that appears in the ribbon bar to show the available settings.

The following options are available:

1. Select the Enable Polling check box to allow polling on the Track. If this check box is not selected, the system ignores the track when looking for new jobs.

KEY FACT:

If Enable Polling is not selected on a Track, a job is created the first time the Track is processed, and the Track is processed once. After that, it is ignored by Track polling. Events can also push recipients through Tracks, and this will occur regardless of whether Track Polling in enabled. Enable Polling is therefore NOT a way to turn off all activity in a track.

2. Poll every: A setting in seconds telling the system how often the track should process.
3. Occurrences: These options are redundant and there are plans to remove them in a future release.

Do Immediately

The default time item on a new track. By default, tracks are polled every 10 minutes (600 seconds). This interval can be adjusted using the polling settings on the Attributes tab. It is possible, and based on campaign design desirable, to configure a different polling interval for every track.

The ‘Do Immediately’ time item will only ever pass a recipient through a track once without them leaving the Track and coming back in as a new instance. Therefore, if someone is processed by a track with a ‘Do Immediately’, they pass to the “bottom” of the track and will never leave. Use a Go To tool or an Events/Trigger to ensure recipients move on.

KEY QUESTION

What happens when the campaign or the system is paused and re-started? When a campaign is manually paused and restarted from the application, potentially for editing, it will begin processing again from the Start track. This is because, on restart, all existing jobs for a campaign are cleared from the jobs queue. The reasoning behind this clearing of jobs is that the scheduling information for tracks may have been altered while the campaign was paused/edited and this could lead to jobs being executed at the wrong time if non-executed jobs remain. The overnight pause/resume process works differently in that existing jobs are left in the queue so that processing continues where it left off. This allows existing scheduled jobs to keep the 'history' of when they were last executed. It is presumed there will have been no changes to the campaign during the pause/resume process so it is safe to continue from where we were.

After A Period

‘After a Period’ is the other time item that uses polling Attributes, again with a default poll every 600 seconds. This should be adjusted as required. Depending on how recipients enter the track, it is quite possible that no new audience members will meet the elapsed time criteria, so while there will still be an overhead associated with audience and track population checks, no members may actually be processed. The job is unlikely to take a lot of time to process. The After a Period time item works from the time that the individual recipient entered the track. Therefore, the period is applied to each recipient on a recipient basis not globally.

KEY QUESTION

When would I use the After a Period? These are likely to be used when an automatic follow up is required at a recipient level. Example: An initial Track sends a recipient an email then using a ‘Go To’ Track tool, sends them to a new Track with an After a Period set to two weeks. This can be considered as a holding track. Two weeks later, the system can automatically send them a follow up based on how long that recipient has been in the Track.

KEY QUESTION

What happens if the campaign or the system is paused? When paused, all polling activity stops. When the campaign system re-starts, the polling frequency will pick back up to see if a recipient has hit the ‘After a Period’ setting, and if they have, a job is created.

On A Schedule

The scheduling settings of this track are checked by the SQL Jobs Service which determines if a job is required. Since the schedule is specified explicitly, there is no ability to change the polling interval and the UI does not display the tab.

KEY FACT:

This tool has one unique property. It is the only tool that can pass the same recipient through the same Track multiple times, without them having to leave the track and come back. Example - The tool is scheduled to run at 10am every day. The Track has a ‘Waterfall’ tool with some criteria and the recipients pass through the Track at 10am on Day 1. Some recipients meet the criteria and are processed by the ‘Waterfall’ tool and passed to a new Track. Some pass through but have no actions. On any other time Item type, they would sit in the track indefinitely, having been processed once. With the ‘On a Schedule’, they would pass through the ‘Waterfall’ again at 10am on Day 2, i.e. their processed flag is over-written by the ‘On a Schedule’. This functionality can be very useful on initial Tracks to enter the campaign i.e. they enter a campaign if a variable value changes they need to be in the audience but can cycle through the START Track.

KNOWN ISSUE:

On a Schedule tool operates differently when multiple schedules are set in a day compared to if the schedule is daily or less frequent. If a schedule has an intra-day frequency i.e. every 4 hours, it will assume an “At least” setting and run straight away, and if one is missed, adjust the time for subsequent schedules. This functionality is under review. Please refer to release notes of future releases for a visibility of any changes. Example Intra-Day behavior – A schedule is set every 4 hours from 10am, so will run at 10am, 2pm, 6pm etc. The system is paused at 5:30pm so the 6pm is missed. The system re-starts at 7:30am. It will try to run the 6pm schedule straight away as it is overdue, but will then run 4 hourly from 7:30am. Example Inter-Day behavior – A schedule is set every 1 day at 6pm. The system is paused at 5:30pm so the 6pm is missed. The system re-starts at 7:30am, it will ignore the 6pm schedule from the day before and will next run at 6pm that day. NOTE: There is a 1 hour grace period on Inter-Day schedules. In the above example, if the system is paused at 5:30pm and re-started at 6:15pm, the 6pm scheduled job would run as it is within 1 hour of the scheduled time, but at 7:15pm, i.e. more than 1 hour from the specified scheduled, it would not run until 6pm the next day. This does not adjust the next schedule.

On A Date

The scheduling criteria of this Track is checked by the SQL Jobs Service when it runs, to determine if a job is required. Since the schedule is specified explicitly, there is no need to change the polling interval and the UI does not display the tab. This node has no repeatability so any missed dates will simply make this tool redundant.

The default Polling frequency of Do Immediately and After a Period Tracks is 600 seconds (10 minutes). This is a setting targeted and campaigns being able to react in reasonable time scales, to any events or changes in data.

This setting can be altered in the Applications Administration section under Client Settings.

Considering all the information, there are a few points that a worth summarizing:

Jobs are processed in the order they are submitted which is left to right from the START track.

All time items except the ‘On a Schedule’ will process recipient once, so don’t leave people on the bottom of tracks if you don’t have any events to move them.

Do not be too aggressive with polling if not required.

End or Termination Tracks: These tracks are often placed at the “end” of a campaign and are used to collect recipients who have completed the campaign. As these tracks have no actions, polling should be disabled on these tracks as polling them serves no purpose but will add to the job queue.

‘Start’ Tracks: Start tracks are fully functional and can be used for any purpose. The only limitation is that they cannot be re-named. If you create a complex campaign and want to name the Track relating to its activity, the START track may be better utilized for variable setting only, but this is entirely by choice.

There are considerations with the process of Events that have a considerable impact on the campaign design. Understanding these are useful for marketers in the creation of campaigns.

There are other options in the campaign that help with campaign design.

What do the Campaign dates control?

There are three configurable dates for a campaign:

1. Start date: Before this date, the campaign does not begin processing even if started.
2. Stop Process: After this date, the campaign no longer performs any polling. Events are still captured for reporting, but no Triggers are fired.
3. End Date: After this date, the campaign no longer performs any polling. Events are still captured for reporting, but no Triggers are fired.

What should I do when the Campaign is finished?

Campaigns store the position and variable values for all recipients in a database table called Campaign State. Having a high volume of State tables can have a performance impact on the system, so it is important that users show some ownership of campaigns and housekeeping.

Once a campaign has completed and is past its End Date, click the Remove State button on the Overview tab to remove all state data related to the campaign.

KEY FACT:

It is the responsibility of the user to ‘Remove State’ on their campaigns once complete. Failure to do so impacts system performance by leaving a high volume of unused and unwanted data in the system.

This section looks at some simple campaign examples and provides advice on Polling.

Basic Single Execution Campaign

A single deployment campaign with no event processing. The audience is defined at the start of campaign and will not change and a single tactic will be used.

Suggestion would be just to use the START track with an ‘On a Schedule’ time item to start the campaign at the required time with no re-occurrence of the schedule.

End Date set for the next day (to ensure all processing is given a chance to finish)

Remember to Remove State after execution.

Start Track

Time Item	On a Schedule run once a day.
Track Tool - Implement Tactic	Initial Creative (Email or File output)
Event/Triggers Tool	None

Basic Engagement Campaign

A basic single contact campaign, that runs continually. It executes once a day to process any new audience members but no Event configuration is involved. A presumption here is that any data changes occur in the overnight load process, and not during the Business as Usual workday.

This campaign has a single Tactic/Creative execution. Suggestion here is to use two tracks, the default Start Track and an End Track.

Tactic implementation occurs in the START track, processed recipients are then moved to the End Track.

KEY QUESTION:

Will recipients ever want to pass through this campaign again, for example a re-engagement campaign? If so, it may be required to send them back to the start Track and allow the Audience rule to re-include them if they meet the criteria again.

Time item ‘On a Schedule’ to run once a day, at an appropriate time. It is useful to avoid times when Ad Hoc high volume single execution campaigns might be running (due to campaign locking).

Remember to turn Track Polling off for the End Track.

Start Track

Time Item	On a Schedule run once a day
Track Tool - Implement Tactic	Initial Email
Track Tool - Go to Track	End Track
Event/Triggers Tool	None

End Track

Time Item	Do Immediately - Disable Polling
Track Tool	None
Event/ Set Up	None

Basic Single Contact and Follow up Campaign

An initial contact email based on an Audience rule, and then an Event based Follow up communication to complete the campaign. Initial contact Audience not presumed to change intra-day.

This can include three Tracks, called Start, Follow Up and End configured and two Tactics called ’Initial Email‘ and ’Follow-Up Email‘ as follows:

Start Track

Time Item	On a Schedule run once a day
Track Tool - Implement Tactic	Initial Email
Track Tool - Go to Track	End Track
Event/Triggers Tool	None

Follow Up Track

Time Item	Do Immediately - Disable Polling **not required as this Track will execute with Event Polling/Chaining
Track Tool	None
Event/ Set Up	Trigger on Email Open
Trigger Tool – Implement Tactic	Follow Up Email
Trigger Tool – Go To Track	End

End Track

Time Item	Do Immediately - Disable Polling
Track Tool	None
Event/ Set Up	None
Trigger Tool – Implement Tactic	None